TST-50: Property-based and adversarial input tests#818
Conversation
Test RFC 3986 edge-case URLs, dangerous schemes (javascript:, data:, file:), SSRF vectors (localhost, metadata endpoints), URLs with credentials, malformed URLs, and Unicode in URLs against the webhook endpoint. All verify NO 500 responses.
Test malformed proposal JSON (wrong types, missing fields, extra unknown fields, enum out of range), adversarial summary strings, boundary-length summaries, and expiry minutes boundaries. Documents two known 500 bugs in skip-annotated tests: XSS in actionType and nested parameter JSON.
Exercise capture/inbox with boundary text lengths, null bytes, control characters, Unicode edge cases, nested JSON, XSS/injection payloads, adversarial optional fields, malformed JSON bodies, and empty/nonexistent board IDs. All verify stored-verbatim semantics and no 500s.
Deterministic-seed random generator sends 100 board, 100 capture, 50 card, 50 search, 50 column, and 50 label requests with random content mixing ASCII, CJK, Arabic, combining diacriticals, and control characters. Verifies no 500 responses and no exposed stack traces.
Property-based tests verifying CaptureProvenanceV1, CapturePayloadV1, and CreateCaptureItemDto survive JSON serialize/deserialize with identity. Exercises adversarial strings in provenance fields, DateTimeOffset boundary values, and arbitrary JSON deserialization safety.
Test GUID format variations (empty, uppercase, lowercase, mixed-case), DateTimeOffset boundaries (min, max, epoch, Y2038, Y2K), string edge cases (empty, whitespace, max-length, over-max), position boundaries (negative, zero, MAX_INT), WipLimit boundaries, and entity Touch property across all entity types.
Test proposal operation parameter round-trips, capture provenance round-trips with adversarial strings, webhook URL display safety for dangerous schemes, numeric boundary values (MAX_SAFE_INTEGER, Infinity, NaN), and capture text with binary-like data and performance bounds.
Remove unused RandomGuidString() method from No500sMetaTest. Improve CaptureProvenanceArb generator to vary string fields (promptVersion, provider, model, sourceSurface) across runs instead of using fixed values.
Self-review findingsReviewed all 7 new test files. Issues found and addressed: Fixed
Accepted (not bugs)
Bugs found (pre-existing, not introduced by this PR)
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 61924ae6e5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| new StringContent("{}", Encoding.UTF8, "application/json")); | ||
| } | ||
|
|
||
| if ((int)response.StatusCode >= 500) |
There was a problem hiding this comment.
Fail the meta-test when a 500 response is returned
In AuthenticatedEndpoints_NeverReturn500WithStackTrace, the test only inspects the body if status is 500, but never asserts that 500 is forbidden, so a plain 500 response without stack-trace text still passes. That creates a false negative for the exact regression this meta-test is supposed to catch (NeverReturn500), allowing server errors on these endpoints to slip through CI unnoticed.
Useful? React with 👍 / 👎.
| new CreateOutboundWebhookSubscriptionDto(endpointUrl)); | ||
|
|
||
| // Valid HTTPS URLs should either succeed or be accepted | ||
| ((int)response.StatusCode).Should().BeLessThan(500); |
There was a problem hiding this comment.
Assert success for valid webhook URLs
CreateWebhook_WithValidUrl_Succeeds currently checks only status < 500, which means a regression that rejects all valid HTTPS webhook URLs with 4xx would still pass this test. Because this method is the only positive-path check in this file, it should assert a success/acceptance status to actually protect valid URL behavior.
Useful? React with 👍 / 👎.
|
|
||
| const elapsed = performance.now() - start | ||
| // Should complete well under 1 second | ||
| expect(elapsed).toBeLessThan(1000) |
There was a problem hiding this comment.
Remove hard wall-clock threshold from unit test
The test enforces elapsed < 1000ms for JSON serialize/parse of a 50K string, which depends on CI machine load and runtime variance rather than functional correctness. On slower or contended runners this can fail intermittently even when behavior is correct, introducing flaky test failures into the suite.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive suite of adversarial and property-based tests to improve the robustness of the API and domain logic. The changes include new test classes for capture, proposal operations, and webhook URL validation, alongside fuzz tests for serialization round-trips and boundary-value tests for domain entities. My feedback highlights concerns regarding thread safety in random number generation, the need for null byte coverage in fuzzing, the removal of unused code, and the optimization of property-based tests by removing unnecessary sleeps.
|
|
||
| // ─────────────────────── Random content generators ─────────────────────── | ||
|
|
||
| private static readonly Random Rng = new(42); // deterministic seed for reproducibility |
There was a problem hiding this comment.
The System.Random class is not thread-safe. Since this Rng instance is static and shared across the class, concurrent execution of tests (which can occur in xUnit depending on configuration) may lead to race conditions, resulting in corrupted state or zero values. Consider using lock (Rng) when accessing it, or using a thread-safe alternative if parallelism is expected.
| var category = Rng.Next(10); | ||
| chars[i] = category switch | ||
| { | ||
| 0 => (char)Rng.Next(1, 32), // control chars (skip null) |
There was a problem hiding this comment.
Adversarial tests should ideally include null bytes (\0), as they are a common source of vulnerabilities and crashes in string handling, database drivers, and serialization logic. Since the PR summary mentions exercising null bytes, they should be included in this randomized sweep as well.
0 => (char)Rng.Next(0, 32), // control chars (including null)| // ─────────────────────── Board creation sweep ─────────────────────── | ||
|
|
||
| [Fact] | ||
| public async Task BoardCreation_100RandomPayloads_Never500() |
| Guid.NewGuid(), | ||
| TriageRunId: Guid.NewGuid(), | ||
| ProposalId: Guid.NewGuid(), | ||
| PromptVersion: pv, |
There was a problem hiding this comment.
Using DateTimeOffset.UtcNow inside an FsCheck generator makes the property test non-deterministic. If a failure occurs, the reproduction of that specific failure using the same seed might fail because the timestamp will have changed. It is better to generate the DateTimeOffset using FsCheck's built-in generators to ensure full reproducibility.
| { | ||
| case "board": | ||
| var board = new Board("Test"); | ||
| initialUpdatedAt = board.UpdatedAt; | ||
| Thread.Sleep(1); | ||
| board.Update(name: "Updated"); | ||
| afterUpdatedAt = board.UpdatedAt; | ||
| break; | ||
| case "card": | ||
| var card = new Card(Guid.NewGuid(), Guid.NewGuid(), "Title"); | ||
| initialUpdatedAt = card.UpdatedAt; | ||
| Thread.Sleep(1); | ||
| card.Update(description: "Updated"); | ||
| afterUpdatedAt = card.UpdatedAt; | ||
| break; | ||
| case "column": | ||
| var col = new Column(Guid.NewGuid(), "Col", 0); | ||
| initialUpdatedAt = col.UpdatedAt; | ||
| Thread.Sleep(1); | ||
| col.SetPosition(1); | ||
| afterUpdatedAt = col.UpdatedAt; | ||
| break; | ||
| case "label": | ||
| var label = new Label(Guid.NewGuid(), "Label", "#FF0000"); | ||
| initialUpdatedAt = label.UpdatedAt; | ||
| Thread.Sleep(1); | ||
| label.Update(name: "Updated"); | ||
| afterUpdatedAt = label.UpdatedAt; | ||
| break; | ||
| default: | ||
| throw new InvalidOperationException($"Unknown entity type: {entityType}"); |
There was a problem hiding this comment.
The use of Thread.Sleep(1) inside a property-based test (which runs 200 iterations by default) significantly slows down the test suite. Furthermore, DateTimeOffset.UtcNow resolution on some operating systems can be as high as 15ms, so a 1ms sleep does not guarantee that the timestamp will advance. Since the assertion on line 385 uses BeOnOrAfter, the test will pass even if the timestamps are identical. Removing the sleeps will improve performance without affecting the validity of the current assertion.
switch (entityType)
{
case "board":
var board = new Board("Test");
initialUpdatedAt = board.UpdatedAt;
board.Update(name: "Updated");
afterUpdatedAt = board.UpdatedAt;
break;
case "card":
var card = new Card(Guid.NewGuid(), Guid.NewGuid(), "Title");
initialUpdatedAt = card.UpdatedAt;
card.Update(description: "Updated");
afterUpdatedAt = card.UpdatedAt;
break;
case "column":
var col = new Column(Guid.NewGuid(), "Col", 0);
initialUpdatedAt = col.UpdatedAt;
col.SetPosition(1);
afterUpdatedAt = col.UpdatedAt;
break;
case "label":
var label = new Label(Guid.NewGuid(), "Label", "#FF0000");
initialUpdatedAt = label.UpdatedAt;
label.Update(name: "Updated");
afterUpdatedAt = label.UpdatedAt;
break;
default:
throw new InvalidOperationException($"Unknown entity type: {entityType}");
}There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 86e2b02c1e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| throw new InvalidOperationException($"Unknown entity type: {entityType}"); | ||
| } | ||
|
|
||
| afterUpdatedAt.Should().BeOnOrAfter(initialUpdatedAt); |
There was a problem hiding this comment.
Require UpdatedAt to move forward after entity updates
This property claims to verify that updates advance UpdatedAt, but afterUpdatedAt.Should().BeOnOrAfter(initialUpdatedAt) also passes when timestamps are equal. If a regression stops touching UpdatedAt in Board.Update/Card.Update/Column.SetPosition/Label.Update, this test still passes with afterUpdatedAt == initialUpdatedAt, so it cannot catch the intended failure mode.
Useful? React with 👍 / 👎.
| try { | ||
| JSON.parse(param) | ||
| } catch { | ||
| // Expected - the frontend should handle this gracefully | ||
| } |
There was a problem hiding this comment.
Remove swallowed parse errors from malformed-params test
The assertion is ineffective because JSON.parse(param) is wrapped in an inner try/catch that suppresses all errors before expect(...).not.toThrow() runs. That means this test passes regardless of malformed input behavior and cannot detect regressions in how malformed operation parameters are handled.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Pull request overview
Adds broad property-based and adversarial input testing across backend API/domain/application and frontend to ensure hostile/malformed inputs don’t crash the system (especially avoiding 500s), and to validate serialization/query boundary behavior.
Changes:
- Added new frontend fast-check adversarial/property tests for proposal/capture/webhook URL/numeric edge cases.
- Added backend FsCheck/Xunit property tests for capture provenance/payload serialization round-trips and domain/query boundary values.
- Added backend API adversarial tests and a randomized “no 500s” meta-sweep across multiple endpoints.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| frontend/taskdeck-web/src/tests/property/adversarialData.spec.ts | New frontend property/adversarial test suite for JSON round-trips, URL/text/numeric edge cases |
| backend/tests/Taskdeck.Domain.Tests/PropertyBased/QueryBoundaryValueTests.cs | New domain boundary-value tests for GUID/DateTime/string/position/WIP limits + UpdatedAt behavior |
| backend/tests/Taskdeck.Application.Tests/Fuzz/CaptureProvenanceRoundTripFuzzTests.cs | New FsCheck round-trip and deserialization-safety fuzz tests for capture payload/provenance DTOs |
| backend/tests/Taskdeck.Api.Tests/WebhookUrlAdversarialTests.cs | New API adversarial tests for webhook URL validation and robustness (no 500s) |
| backend/tests/Taskdeck.Api.Tests/ProposalOperationsAdversarialTests.cs | New API adversarial tests for proposal creation with malformed/hostile operation payloads |
| backend/tests/Taskdeck.Api.Tests/No500sMetaTest.cs | New randomized cross-endpoint sweep intended to ensure “no 500s” / no stack-trace leakage |
| backend/tests/Taskdeck.Api.Tests/CaptureAdversarialTests.cs | New API adversarial tests for capture endpoint with binary/control/unicode/injection/malformed JSON inputs |
Comments suppressed due to low confidence (1)
backend/tests/Taskdeck.Api.Tests/No500sMetaTest.cs:265
- Despite the class name/summary (“No500s”), this test will still pass if an endpoint returns 500 as long as the body doesn’t contain the checked substrings. If the intent is truly “never 500”, add an assertion that StatusCode < 500; otherwise consider renaming the test/class to reflect that it’s specifically a “no stack trace leakage” check.
[InlineData("GET", "/api/notifications")]
public async Task AuthenticatedEndpoints_NeverReturn500WithStackTrace(string method, string path)
{
await EnsureAuthenticatedAsync();
HttpResponseMessage response;
if (method == "GET")
{
response = await _client.GetAsync(path);
}
else
{
response = await _client.PostAsync(path,
new StringContent("{}", Encoding.UTF8, "application/json"));
}
if ((int)response.StatusCode >= 500)
{
var body = await response.Content.ReadAsStringAsync();
body.Should().NotContain("StackTrace",
$"{method} {path} returned 500 with stack trace exposed");
body.Should().NotContain("at Taskdeck.",
$"{method} {path} returned 500 with internal exception details exposed");
}
}
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| it('malformed operation parameters string does not throw when parsed cautiously', () => { | ||
| const malformedParams = [ | ||
| '', | ||
| 'not json', | ||
| '{', | ||
| '[', | ||
| '{"unclosed": ', | ||
| 'null', | ||
| '12345', | ||
| '<xml>data</xml>', | ||
| ] | ||
|
|
||
| for (const param of malformedParams) { | ||
| expect(() => { | ||
| try { | ||
| JSON.parse(param) | ||
| } catch { | ||
| // Expected - the frontend should handle this gracefully | ||
| } | ||
| }).not.toThrow() | ||
| } |
There was a problem hiding this comment.
This test is effectively a no-op: JSON.parse errors are caught and swallowed inside the expectation, so the outer expect().not.toThrow() will always pass regardless of input. Consider exercising the actual parsing/validation helper the app uses (or add one) and assert a concrete outcome (e.g., returns null/Result/error state) for malformed parameter strings.
| it('capture text with binary-like data should serialize', () => { | ||
| // Simulate various binary-like string patterns | ||
| const binaryPatterns = [ | ||
| '\x00\x01\x02\x03\x04\x05', | ||
| String.fromCharCode(...Array.from({ length: 256 }, (_, i) => i)), | ||
| '\uFFFE\uFFFF', // non-characters | ||
| '\uD800', // lone surrogate (will be replaced in JSON) | ||
| ] | ||
|
|
||
| for (const pattern of binaryPatterns) { | ||
| expect(() => { | ||
| try { | ||
| const json = JSON.stringify({ text: pattern }) | ||
| JSON.parse(json) | ||
| } catch { | ||
| // Some patterns may fail JSON serialization, which is expected | ||
| } | ||
| }).not.toThrow() | ||
| } |
There was a problem hiding this comment.
Same issue as above: the try/catch inside expect().not.toThrow() swallows any JSON.stringify/JSON.parse failures, so this assertion will always pass and doesn't validate behavior. If the goal is resilience, assert that serialization succeeds for the supported patterns, and explicitly assert failure/handling behavior for patterns you expect to be rejected.
| it('very long capture text (50K chars) should not freeze', () => { | ||
| const longText = 'x'.repeat(50_000) | ||
| const start = performance.now() | ||
|
|
||
| const json = JSON.stringify({ text: longText }) | ||
| JSON.parse(json) | ||
|
|
||
| const elapsed = performance.now() - start | ||
| // Should complete well under 1 second | ||
| expect(elapsed).toBeLessThan(1000) | ||
| }) |
There was a problem hiding this comment.
This uses a hard 1s wall-clock threshold (performance.now()) which is prone to CI flakiness under load and doesn't really measure UI “freeze” (it only measures JSON stringify/parse). Consider removing the timing assertion (just ensure it completes), or make it much more tolerant and/or run it as a benchmark-style test outside the main suite.
| public void Card_DueDate_AcceptsNull() | ||
| { | ||
| var card = new Card(Guid.NewGuid(), Guid.NewGuid(), "Title"); | ||
| card.Update(dueDate: null); | ||
| card.DueDate.Should().BeNull(); | ||
| } |
There was a problem hiding this comment.
This test doesn’t actually verify that a due date can be cleared: the card starts with DueDate == null, and Card.Update(dueDate: null) is a no-op with the current Card.Update implementation (it only assigns when dueDate.HasValue). To test “accepts null/clear”, first set a non-null due date, then call Update(dueDate: null) and assert it becomes null (or rename the test if clearing isn’t supported).
| { | ||
| var body = await response.Content.ReadAsStringAsync(); | ||
| body.Should().NotContain("System.", "500 response should not contain stack traces"); | ||
| body.Should().NotContain("at Taskdeck.", "500 response should not contain stack traces"); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // ─────────────────────── Capture creation sweep ─────────────────────── | ||
|
|
There was a problem hiding this comment.
The stack-trace body check is unreachable: the BeLessThan(500) assertion will already fail the test before the subsequent if (StatusCode >= 500) block can run. If you want to assert both “not 500” and “no stack trace if 500”, capture the body first and/or use an AssertionScope so both checks can be evaluated.
| var body = await response.Content.ReadAsStringAsync(); | ||
| body.Should().NotContain("System.", "500 response should not contain stack traces"); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // ─────────────────────── Card creation sweep ─────────────────────── | ||
|
|
||
| [Fact] |
There was a problem hiding this comment.
Same unreachable-code issue here: the BeLessThan(500) assertion will throw before the if (StatusCode >= 500) branch can execute, so the response-body redaction check never runs. Consider restructuring to always read/assert on the body when status >= 500 (or use AssertionScope).
| private static readonly Random Rng = new(42); // deterministic seed for reproducibility | ||
|
|
||
| private static string RandomString(int minLen, int maxLen) | ||
| { | ||
| var length = Rng.Next(minLen, maxLen + 1); | ||
| var chars = new char[length]; | ||
| for (int i = 0; i < length; i++) | ||
| { | ||
| // Mix of ASCII, unicode, and control chars | ||
| var category = Rng.Next(10); | ||
| chars[i] = category switch | ||
| { | ||
| 0 => (char)Rng.Next(1, 32), // control chars (skip null) | ||
| 1 => (char)Rng.Next(0x4E00, 0x4F00), // CJK | ||
| 2 => (char)Rng.Next(0x0600, 0x0700), // Arabic | ||
| 3 => (char)Rng.Next(0x0300, 0x0370), // combining diacriticals | ||
| 4 => (char)Rng.Next(0x2000, 0x2070), // general punctuation / special | ||
| _ => (char)Rng.Next(0x20, 0x7F), // printable ASCII | ||
| }; | ||
| } | ||
| return new string(chars); | ||
| } |
There was a problem hiding this comment.
Rng is a shared static Random consumed across multiple [Fact]s. Because xUnit does not guarantee test execution order, the sequence of “random” values (and therefore failures) can change between runs despite the fixed seed. To keep runs reproducible, instantiate a new Random(42) per test (or reset the RNG at the start of each test), rather than sharing a single static instance.
| /// <summary> | ||
| /// Adversarial tests for webhook endpoint URL validation. | ||
| /// Verifies that dangerous, malformed, and RFC 3986 edge-case URLs | ||
| /// are rejected with 4xx — never 5xx. | ||
| /// </summary> | ||
| public class WebhookUrlAdversarialTests : IClassFixture<TestWebApplicationFactory> | ||
| { | ||
| private readonly TestWebApplicationFactory _factory; | ||
| private readonly HttpClient _client; | ||
| private bool _isAuthenticated; | ||
| private Guid? _boardId; | ||
|
|
||
| public WebhookUrlAdversarialTests(TestWebApplicationFactory factory) | ||
| { | ||
| _factory = factory; | ||
| _client = factory.CreateClient(); | ||
| } | ||
|
|
||
| private async Task EnsureAuthenticatedAsync() | ||
| { | ||
| if (_isAuthenticated) return; | ||
| await ApiTestHarness.AuthenticateAsync(_client, "webhook-adversarial"); | ||
| _isAuthenticated = true; | ||
| } | ||
|
|
||
| private async Task<Guid> EnsureBoardAsync() | ||
| { | ||
| if (_boardId.HasValue) return _boardId.Value; | ||
| await EnsureAuthenticatedAsync(); | ||
| var board = await ApiTestHarness.CreateBoardAsync(_client, "webhook-test"); | ||
| _boardId = board.Id; | ||
| return board.Id; | ||
| } | ||
|
|
||
| // ─────────────────────── RFC 3986 edge-case URLs ─────────────────────── | ||
|
|
||
| public static IEnumerable<object[]> AdversarialWebhookUrls() | ||
| { | ||
| // Dangerous URL schemes | ||
| yield return new object[] { "javascript:alert(1)" }; | ||
| yield return new object[] { "data:text/html,<script>alert(1)</script>" }; | ||
| yield return new object[] { "data:text/html;base64,PHNjcmlwdD5hbGVydCgxKTwvc2NyaXB0Pg==" }; | ||
| yield return new object[] { "vbscript:MsgBox(1)" }; | ||
| yield return new object[] { "file:///etc/passwd" }; | ||
| yield return new object[] { "ftp://evil.com/payload" }; | ||
|
|
||
| // URLs with credentials (userinfo component) | ||
| yield return new object[] { "https://admin:password@evil.com/webhook" }; | ||
| yield return new object[] { "http://user:pass@127.0.0.1/callback" }; | ||
|
|
||
| // Internal/localhost URLs (SSRF vectors) | ||
| yield return new object[] { "http://localhost/webhook" }; | ||
| yield return new object[] { "http://127.0.0.1/webhook" }; | ||
| yield return new object[] { "http://[::1]/webhook" }; | ||
| yield return new object[] { "http://0.0.0.0/webhook" }; | ||
| yield return new object[] { "http://169.254.169.254/latest/meta-data/" }; | ||
| yield return new object[] { "http://metadata.google.internal/" }; | ||
|
|
||
| // Malformed URLs | ||
| yield return new object[] { "" }; | ||
| yield return new object[] { " " }; | ||
| yield return new object[] { "not-a-url" }; | ||
| yield return new object[] { "://missing-scheme" }; | ||
| yield return new object[] { "http://" }; | ||
| yield return new object[] { "http:///no-host" }; | ||
|
|
||
| // URLs with injection payloads | ||
| yield return new object[] { "https://evil.com/webhook?q='; DROP TABLE webhooks; --" }; | ||
| yield return new object[] { "https://evil.com/webhook#<script>alert(1)</script>" }; | ||
| yield return new object[] { "https://evil.com/\u0000null-byte" }; | ||
| yield return new object[] { "https://evil.com/\r\nHeader-Injection: true" }; | ||
|
|
||
| // URLs with unicode | ||
| yield return new object[] { "https://evil.com/\u200Bhidden" }; | ||
| yield return new object[] { "https://evil.com/\u202Efdp.exe" }; | ||
| yield return new object[] { "https://xn--n3h.com/webhook" }; // punycode emoji domain | ||
|
|
||
| // Extremely long URL | ||
| yield return new object[] { "https://example.com/" + new string('a', 10_000) }; | ||
|
|
||
| // URL with port boundaries | ||
| yield return new object[] { "https://example.com:0/webhook" }; | ||
| yield return new object[] { "https://example.com:99999/webhook" }; | ||
| yield return new object[] { "https://example.com:-1/webhook" }; | ||
| } | ||
|
|
||
| [Theory] | ||
| [MemberData(nameof(AdversarialWebhookUrls))] | ||
| public async Task CreateWebhook_WithAdversarialUrl_NeverReturns500(string endpointUrl) | ||
| { | ||
| var boardId = await EnsureBoardAsync(); | ||
|
|
||
| var response = await _client.PostAsJsonAsync( | ||
| $"/api/boards/{boardId}/webhooks", | ||
| new CreateOutboundWebhookSubscriptionDto(endpointUrl)); | ||
|
|
||
| ((int)response.StatusCode).Should().BeLessThan(500, | ||
| $"Webhook creation returned 500 for URL: {endpointUrl}"); | ||
| } |
There was a problem hiding this comment.
The XML doc says adversarial URLs “are rejected with 4xx — never 5xx”, but the assertion only checks < 500, so 2xx responses (accepting a dangerous URL) would still pass. If the requirement is rejection, assert a 4xx response (and consider environment-specific exceptions like localhost being allowed in Development per OutboundWebhooksApiTests.cs:104-113).
| [Theory] | ||
| [InlineData("https://example.com/webhook")] | ||
| [InlineData("https://hooks.slack.com/services/T00/B00/xxxx")] | ||
| [InlineData("https://example.com:8443/api/webhook")] | ||
| public async Task CreateWebhook_WithValidUrl_Succeeds(string endpointUrl) | ||
| { | ||
| var boardId = await EnsureBoardAsync(); | ||
|
|
||
| var response = await _client.PostAsJsonAsync( | ||
| $"/api/boards/{boardId}/webhooks", | ||
| new CreateOutboundWebhookSubscriptionDto(endpointUrl)); | ||
|
|
||
| // Valid HTTPS URLs should either succeed or be accepted | ||
| ((int)response.StatusCode).Should().BeLessThan(500); | ||
| } |
There was a problem hiding this comment.
Test name says “Succeeds”, but the assertion only checks that the response is not a 5xx. Either tighten the assertion to the expected success status (e.g., 201 Created) or rename the test to reflect the weaker contract (“never 500”).
- Remove unnecessary backslash escapes in JSON string constant (no-useless-escape) - Replace hard wall-clock threshold with functional round-trip assertion - Replace swallowed-exception tests with explicit throw/value assertions - Fix binary data test to verify actual round-trip instead of swallowing errors
- Replace static Random with per-test CreateRng() for thread safety - Add null byte (\0) to adversarial character set - Make AuthenticatedEndpoints test actually fail on 500 responses - Remove unreachable stack-trace checks after BeLessThan assertion
- Eliminates Thread.Sleep(1) * 200 iterations performance cost - Uses BeAfter instead of BeOnOrAfter for strict timestamp advancement - Uses 16ms sleep to exceed Windows timer resolution
Review findings and fixesInvestigated CI failures and all bot review comments. All issues addressed in follow-up commits. CI fix (Priority 1)
Bot review issues addressed
Fresh adversarial review findings (also fixed)
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 746bbc880f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ((int)response.StatusCode).Should().BeLessThan(500, | ||
| $"Webhook creation returned 500 for URL: {endpointUrl}"); |
There was a problem hiding this comment.
Assert 4xx rejection for adversarial webhook endpoints
CreateWebhook_WithAdversarialUrl_NeverReturns500 currently only asserts status < 500, so a regression that starts accepting dangerous endpoints (for example javascript:/data:/metadata-host URLs) with 2xx would still pass this test. That makes this adversarial security test a false negative for URL-validation/SSRF regressions; it should assert explicit rejection semantics (4xx), with any intentionally allowed cases split into separate tests.
Useful? React with 👍 / 👎.
Summary
Closes #717
Test plan